Markov process large deviations in τ-topology

Nonequilibrium Markov Processes Conditioned on Large Deviations

Large Deviations for a Class of Nonhomogeneous Markov Chains

Large deviation results are given for a class of perturbed non-homogeneous Markov chains on finite state space which formally includes some stochastic optimization algorithms. Specifically, let {Pn} be a sequence of transition matrices on a finite state space which converge to a limit transition matrix P. Let {Xn} be the associated non-homogeneous Markov chain where Pn controls movement from ti...

Large Deviations for Occupation Measures for Markov Processes

A simple proof of Donsker-Varadhan’s large deviation principle (LDP) for occupation measure of Markov process, valued in R, with the discrete time is given. A proof is based on a new version of Dupui-Ellis’s large deviation principle for two-dimensional occupation measures. In our setting, an existence of the invariant measure does not assumed. This condition is replaced (from point of view of ...

Large deviations of Markov hains indexed by random trees

Large Deviations Bounds for Markov Decision Processes Under General Policies

We consider the empirical state-action frequencies and the empirical reward in weakly communicating finite-state Markov decision processes under general policies. We define a certain polytope and establish that every element of this polytope is the limit of the empirical frequency vector, under some policy, in a strong sense. Furthermore, we show that the probability of exceeding a given distan...